Three-Dimensional Image Warping on Programmable Graphics Hardware

Many image-based rendering systems are based on three-dimensional image warping (3D Warping), which transforms pixels in reference image to destination view. However, the original 3D warping equation, proposed by McMillan and Bishop, is derived under one special coordinate system, making its inability of direct implementation on programmable graphics hardware. In this paper, we revisit the 3D warping process and derive one novel 3D warping formulation using three-dimensional graphics pipeline. Our 3D warping formulation is very simple and uses only one 4×4 matrix. Its evaluation merely involves vector-matrix multiplication, which allows it to be implemented efficiently using vertex shader on modern GPUs. Additionally, our 3D warping formulation is useful for nongraphics problems related to video compression. We will demonstrate that rendering optical flow images can be performed on graphics cards installed on most PCs.